Segmentation of lesions in ultrasound images

ABSTRACT

A method for determining a candidate lesion region in a digital ultrasound medical image of anatomical tissue. The method includes the steps of: accessing the digital ultrasound medical image of anatomical tissue; applying an anisotropic diffusion filter to the ultrasound image to generate a filtered ultrasound image; performing a normalized cut operation on the filtered ultrasound image to partition the filtered ultrasound image into a plurality of regions; and selecting, from the plurality of regions, at least one region as a candidate lesion region.

CROSS REFERENCE TO RELATED APPLICATIONS

Reference is made to, and priority is claimed from, commonly assignedProvisional Patent Application U.S. Ser. No. 60/675,629, entitled“SYSTEMS AND METHODS FOR AUTOMATED ANALYSIS OF LESIONS ON ULTRASOUNDIMAGES”, filed on Apr. 28, 2005 in the names of Huo et al., and which isassigned to the assignee of this application, and incorporated herein byreference.

FIELD OF THE INVENTION

The invention relates to digital image processing of ultrasound images,and more particularly to detecting lesions within such images.

BACKGROUND OF THE INVENTION

Breast cancer is a common cancer of women and a common cause of cancerdeaths. An effective way to improve prognosis and survival rate is earlydetection and treatment of breast cancer. Mammography is an imagingmodality which has provided some effectiveness in the early detection ofclinically occult breast cancer, and is viewed by some to be a primaryimaging modality for breast cancer screening.

Mammography combined with ultrasound (sonography) examination isconsidered by some to be an effective method for early diagnosis ofbreast cancers. As an adjunct to mammography for breast cancer detectionand diagnosis, ultrasound can be used to determine whether a detectedmass from screening mammography is solid or cystic. The characteristicsof the lesion extracted from ultrasound images could also assist indifferentiating between benign and malignant lesions. Refer for exampleto A T Stavros et al., “Solid Breast Nodules: Use Of Sonography ToDistinguish Between Benign And Malignant Lesions”, Radiology, Vol. 196,pp. 123-134, 1995. See also Parker S L, Tong T, Bolden S and Wingo P A.Cancer Statistics. Ca Cancer J Clin 1997; 47:5-27.

Currently, mammography is believed to achieve a reported sensitivity(i.e., a fraction of breast cancers that are detected by mammography) of85%-95%. Despite improved radiographic criteria for differentiatingmalignant from benign lesions of the breast, misclassification oflesions can occur in everyday clinical practice. Refer to the followingreferences.

-   Anant Madabhushi et al, “Combining low-, high-level and empirical    domain knowledge for automated segmentation of ultrasound breast    images”, IEEE Transactions on Medical Imaging, Vol. 22, No. 2, pp    155-169, February 2003.-   Segyeony Joo et al, “Computer-aided diagnosis of solid breast    nodules: use of an artificial neural network based on multiple    sonographic features”, IEEE Transactions on Medical Imaging, Vol.    23, No. 10, pp. 1292-1300, October 2004.-   A. Hammoude, “An Empirical Parameter Selection Method for    Endocardial Border Identification Algorithm”, Computerized Medical    Imaging and Graphics, Vol. 25, pp. 33-45, 2001.-   Bassett L W and Gold R H. Breast Cancer Detection: Mammography and    Other Methods in Breast Imaging. New York, Grune & Stratton, 1987.-   D'Orsi C J and Kopans D B. Mammographic feature analysis. Seminars    in Roentgenology 1993; 28:204-230.-   D'Orsi C J, Swets J A, Pickett R M, Seltzer S E and McNeil B J.    Reading and decision aids for improved accuracy and standardization    of mammographic diagnosis. Radiology 1992; 184:619-622.-   Knutzen A M and Grisvold J J. Likelihood of malignant disease for    various categories of mammographically detected, nonpalpable breast    lesions. Mayo Clin Proc 1993; 68:454-460.-   Sickles E A. Periodic mammographic follow-up of probably benign    lesions: results in 3184 consecutive cases. Radiology 1991;    179:463-468.-   Kopans D B. Breast Imaging. Philadelphia, Lipincott, 1989.    It has been estimated that only 15-30% of mammographic lesions sent    to biopsy are actually malignant. Variability (estimated as 7% to    40%) in positive biopsy rates between individual radiologists has    also been reported. Thus, use of ultrasound images adjunct to    mammography is believed to be increasingly important to reduce the    number of benign cases sent for unnecessary biopsy.

In addition, there is a need for an objective computerizedclassification scheme adapted to differentiate between benign andmalignant masses at the level similar to experienced radiologists topromote improvement in the diagnostic accuracy of less-experiencedradiologists, to further promote the reduction in the number ofunnecessary biopsies for benign lesions.

U.S. Pat. No. 5,984,870 (Giger) is directed to a method and system forthe analysis of a lesion existing in anatomical tissue.

U.S. Patent Application No. 2003/0161513 (Drukker) is directed to theanalysis of lesion shadows in an ultrasound image.

U.S. Pat. No. 6,855,114 (Drukker) is directed to a radial gradient index(RGI) feature in a sonographic image.

A difficulty which has been associated with a computerized system fordetecting and diagnosing breast lesions is the segmentation of thelesion regions from the surrounding tissues. In some systems, thesegmentation is accomplished by manually outlining the lesions using agraphic user interface, for example, U.S. Pat. No. 5,984,870 (Giger).This manual procedure is labor-intensive, can disrupt full automation,and can be prone to human error, inconsistency, and subjectivity.

Accordingly, there exists a need for an automated segmentation modulefor a computerized mammography analysis system. Accurate segmentation ofa breast lesion is an important step to ensure accurate classificationof a detected breast lesion as a benign or malignant lesion. Further,automated segmentation of breast lesions in ultrasound images canimprove the workflow by removing the manual segmentation step.

Several approaches have been proposed to segment ultrasound breastimages for automated diagnosis of breast lesions. See for example,Madabhushi and Metaxas, “Combining low-, high-level and empirical domainknowledge for automated segmentation of ultrasound breast images,” IEEETrans. on Medical Imaging, Vol. 22, No. 2, February 2003, pp. 155-169;and Horsch K, Giger M L, Venta L A and Vyborney C J., “Automaticsegmentation of breast lesions on ultrasound images”, Med Phys 2001;28:1652-1659.

Given that ultrasound images comprise speckle noise and tissue relatedtextures, accurate segmentation task remains as a challenge.

Pixel-based, edge-based, region-based, and model-based segmentationtechniques are known in medical image processing. Some approaches mayhave limitations. For example, pixel-based segmentation techniques tendto have difficulties when there is a significant amount of noise in theimage. Edge-based techniques tend to experience problems when theboundary of the object is not well defined and when the image contrastis poor. Model-based techniques tend to fail when there is a significantamount of variation in the shape and appearance of the object ofinterest. Region-growing techniques require a good seed point (typicallyprovided by manual interaction) and can be subject to errors whenadjoining objects closely match an object of interest in theirappearance. U.S. Patent Application No. 2003/0125621 (Drukker) describesgradient features and region growing methods to segment breast lesionsin ultrasound images.

Accordingly, there exists a need for a method, which overcomes thelimitations of existing methods.

Reference is made to commonly assigned application U.S. Ser. No.10/994,794 (Kodak Docket No. 88819), entitled “DETECTING AND CLASSIFYINGLESIONS IN ULTRASOUND IMAGES”, filed on Nov. 22, 2004 in the names ofLuo et al., and which is assigned to the assignee of this application,and incorporated herein by reference. The Luo et al applicationdescribes a method for detecting a lesion in a digital ultrasound imageof anatomical tissue, the method comprising the steps of: accessing thedigital ultrasound image of anatomical tissue; segmenting spatiallycontiguous pixels in the digital image into a plurality of regions inaccordance with substantially similar intensity patterns; selecting,from the plurality of regions, one or more candidate lesion regionshaving an intensity value lower than a predetermined intensity value;and classifying the one or more candidate lesion regions into at leastone of the following classes: benign, malignant, or unknown.

The present invention provides a lesion detection and segmentationwherein detection and segmentation are automatic. The method examinesthe similarity and dissimilarity in intensity and texture patterns ofregions and identify regions as potential candidates for breast lesions.Thus, the method is less sensitive to the noise and target appearance.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a lesion detectionmethod in which the detection is automatic.

Another object of the present invention is to provide such a lesionsegmentation method in which is relatively insensitive to variations inimage noise and target appearance.

A further object of the present invention is to provide such a method toremove speckle noise to promote accurate segmentation of a lesion forextracting features used by human readers or by a computerclassification method.

These objects are given only by way of illustrative example, and suchobjects may be exemplary of one or more embodiments of the invention.Other desirable objectives and advantages inherently achieved by thedisclosed invention may occur or become apparent to those skilled in theart. The invention is defined by the appended claims.

The present invention provides a method, computer program, and system,in which a lesion region is automatically located within an ultrasoundimage. It is an advantageous effect of the invention that an improvedlesion diagnosis method and apparatus is provided, in which segmentationis automatic and is relatively insensitive to in image noise.

According to one aspect of the present invention, there is provided amethod for determining a candidate lesion region in a digital ultrasoundmedical image of anatomical tissue. The method includes the steps of:accessing the digital ultrasound medical image of anatomical tissue;applying an anisotropic diffusion filter to the ultrasound image togenerate a filtered ultrasound image; performing a normalized cutoperation on the filtered ultrasound image to partition the filteredultrasound image into a plurality of regions; and selecting, from theplurality of regions, at least one region as a candidate lesion region.

According to another aspect of the present invention, there is provideda method for determining a candidate lesion region in a digitalultrasound medical image of anatomical tissue. The method includes thesteps of: (1) accessing the digital ultrasound medical image ofanatomical tissue; (2) applying an anisotropic diffusion filter to theultrasound image to generate a filtered ultrasound image; (3) performinga normalized cut operation on the filtered ultrasound image to partitionthe filtered ultrasound image into a plurality of regions, wherein thenormalized cut is performed by segmenting spatially contiguous pixels inthe filtered ultrasound image into a plurality of regions in accordancewith substantially similar features; (4) merging the plurality ofregions based on pre-determined threshold values; and (5) selecting,from the plurality of regions, at least one region as a candidate lesionregion, wherein the selected candidate lesion region has: (1) anintensity value lower than a pre-determined intensity value and (2)morphological or texture features in accordance with pre-determinedlesion criteria.

BRIEF DESCRIPTION OF THE DRAWINGS

The above-mentioned and other features and objects of this invention andthe manner of attaining them will become more apparent and the inventionitself will be better understood by reference to the followingdescription of an embodiment of the invention taken in conjunction withthe accompanying figures wherein:

FIG. 1 is a block diagram illustrating the steps of an embodiment of themethod in accordance with the present invention.

FIGS. 2A-2F are examples of ultrasound images wherein FIG. 2A shows anoriginal image with a lesion; FIG. 2B shows the image after noisefiltering, FIG. 2C shows the image obtained from a normalized cutsegmentation; FIG. 2D shows the image after region merging; FIG. 2Eshows the smoothed region after morphological operation; and FIG. 2Fshows a final segmentation result.

FIG. 3 generally describes an anisotropic filter technique.

FIG. 4A-4B show examples of ultrasound images with a breast lesion.

FIGS. 5A-5B are ultrasound images wherein FIG. 5A shows a manuallydelineated lesion border and computer delineated lesion border; and FIG.5B shows a mismatched area between the two borders.

FIGS. 6A-6B, 7A-7B, and 8A-8B show examples of original ultrasoundimages (FIGS. 6A, 7A, and 8A) and the ultrasound image withcomputer-determined contours (FIGS. 6B, 7B, and 8B).

DETAILED DESCRIPTION OF THE INVENTION

The following is a detailed description of the preferred embodiments ofthe invention, reference being made to the drawings in which the samereference numerals identify the same elements of structure in each ofthe several figures.

FIG. 1 shows a flowchart generally illustrating the method in accordancewith the present invention. At step 10, an input ultrasound image isaccessed/acquired/provided for analysis. A nonlinear filtering isapplied on the input ultrasound image to remove the noise whilepreserving the edge (step 20). At step 30, the region is segmented.Region selection and merging is accomplished at step 40, and the lesionis segmented at step 50.

More particularly, once the digital ultrasound image of anatomicaltissue is accessed, an anisotropic diffusion filter is used to removenoise from the image. Then, spatially contiguous pixels of the digitalultrasound image are segmented into a plurality of regions in accordancewith substantially similar intensity values and texture features using anormalized cut method. One or more candidate lesion regions are thenselected from the plurality of regions, such that the selected candidatelesion region has an intensity value lower than a pre-determinedintensity value and has morphological and texture features meeting apre-determined lesion criteria.

FIGS. 2A through 2F illustrate the method of the present invention. FIG.2A shows an original ultrasound image of anatomical tissue with a lesion(step 10). FIG. 2B shows the image after noise filtering of step 20.FIG. 2C shows the image obtained from a normalized cut segmentation(step 30). FIG. 2D shows the image after region merging (step 40). FIG.2E shows the smoothed region after morphological operation. FIG. 2Fshows a final segmentation result (step 50).

As mentioned above, a nonlinear filter is applied to the ultrasoundimage at step 20 to remove noise while preserving the edge. At thisstep, an anisotropic diffusion is preferably used as the nonlinearfilter. An advantage of such a filter is that it is adapted to smooththe noise while being able to preserve or possibly enhance edges.

Anisotropic diffusion is well known, and has been described by Peronaand Malik. Refer to Perona et al, “Scale Space and Edge Detection UsingAnisotropic Diffusion”, IEEE Transactions on Pattern Recognition andMachine Intelligence, Vol. 12, No. 6, pp. 629-639, July 1990. Generally,with anisotropic diffusion, an “edge stopping” function is introduced to“stop” the diffusion process for pixels on edges with a strong gradient.A general description of anisotropic diffusion is provided in FIG. 3.

At step 30, a normalized cut (Ncut) is performed to partition the imageinto a number of groups/regions, for example as shown in FIG. 2C.Normalized cuts are known. For example, refer to Jianbo Shi et al,“Normalized Cuts and Image Segmentation”, IEEE Transactions on patterRecognition and Machine Intelligence, Vol. 22, No. 8, pp. 888-905,August 2000. The normalized cut described by Shi et al. formulatessegmentation as a graph-partitioning problem. The normalized cut is ameasure of the goodness of an image partition. A criterion is tomaximize the total dissimilarity between the different groups and thetotal similarity within the groups in an image. This segmentationtechnique employs combinations of different features (such asbrightness, position, windowed histograms, and the like), and Applicantshave recognized its use in applications of different imaging modalities.

Ncut is considered to be an unsupervised segmentation method. For agiven image represented by G=(V, E), with nodes V (feature vector) ofthe graph representing points in the featured space and E representingedges between any two nodes. To partition G into two disjoint sets A andB, the dissimilarity between A and B sets is calculated as shown inEquation 1: $\begin{matrix}{{{cut}\left( {A,B} \right)} = {\sum\limits_{{u \in A},{\upsilon \in B}}{w\left( {u,\upsilon} \right)}}} & \left( {{Eqn}.\quad 1} \right)\end{matrix}$wherein w(u,v), the weight on each edge, is a function of the similaritybetween node u and node v. An optimal cut or partition of these nodes Vinto two disjunctive sets A and B can be reached when cut(A,B) reachesits minimum.

The normalized cut is calculated as in Equation 2: $\begin{matrix}{{N\quad{{cut}\left( {A,B} \right)}} = {\frac{{cut}\left( {A,B} \right)}{{asso}\left( {A,V} \right)} + \frac{{cut}\left( {A,B} \right)}{{asso}\left( {B,V} \right)}}} & \left( {{Eqn}.\quad 2} \right)\end{matrix}$wherein:${{asso}\left( {A,V} \right)} = {\sum\limits_{{u \in A},{t \in V}}{w\left( {u,t} \right)}}$is the total connection from nodes in A to all nodes in graph V. InNcut, cut( ) is normalized by asso( ). The normalization is performed toremove the bias for partitioning out small sets of isolated nodes in thegraph when minimizing cut(A,B).

Features such as intensity-based features (e.g., average gray scale,variations in gray scale), position, windowed histograms andtexture-based features are calculated for each point (node) in an image.Given a set of features, a weighted graph G=(V, E) is generated for animage. The image is partitioned into groups by minimizing Ncut, so as tominimize the similarity between groups and maximize the similaritywithin each group. Thus, the normalized cut is accomplished bysegmenting spatially contiguous pixels in the filtered ultrasound imageinto a plurality of regions in accordance with substantially similarintensity-based features and/or texture-based features. A determinationis then made whether the current partition should be subdivided bychecking the stability of the cut, and determine if Ncut is below apre-determined value. Recursive partitioning of the segmented parts canoccur. FIG. 2C shows the result from the Ncut method. It is noted thatNcut can begin with bipartition or with n by m regions.

Regions selection and merging occurs at step 40. This clustering stepincludes two steps: region merging and selection of lesion region.

The texture property is characterized within each region. Texturefeatures such as the energy, entropy, contrast and homogeneity fromco-occurrence matrix are calculated for each region, as with Equation 3.These textures are selected since the texture properties for lesionregions are differ for non-lesion regions. $\begin{matrix}{{{Entropy} = {- {\sum\limits_{i}{\sum\limits_{j}{{P\left\lbrack {i,j} \right\rbrack}\log\quad{P\left\lbrack {i,j} \right\rbrack}}}}}}{{Energy} = {- {\sum\limits_{i}{\sum\limits_{j}{P^{\quad 2}\left\lbrack {i,j} \right\rbrack}}}}}{{Contrast} = {- {\sum\limits_{i}{\sum\limits_{j}{\left( {i\quad - \quad j} \right)^{2}{P\left\lbrack {i,j} \right\rbrack}}}}}}{{Homogeneity} = {- {\sum\limits_{i}{\sum\limits_{j}\frac{P\left\lbrack {i,\quad j} \right\rbrack}{1\quad + \quad{{i\quad - \quad j}}}}}}}} & \left( {{Eqn}.\quad 3} \right)\end{matrix}$wherein P[i, j] is a two-dimensional Grey scale co-occurrence matrix.

It is noted that lesion regions tend to be dark, with larger entropy andhomogeneity values and a smaller contrast value as shown in FIGS. 4A and4B. Pre-selected/pre-determined threshold values can be employed ascriteria to determine if two regions should be merged.

Two sets of criteria can be employed: one for regions with an averagegray value lower than a value G₀, and one for regions with an averagegray value higher than G₀.

For regions with an average gray value less than G₀, less strictcriteria are applied on the similarity in texture measures between thetwo adjacent regions. The difference between the two regions allowed isdifferent for each texture feature.

For regions with an average gray value greater than G₀, tighter criteriaare applied on the similarity in texture measures between the twoadjacent regions.

FIG. 2D shows the image with merged regions based on texture analysis.

With regard to region selection, a calculation is made of the followingfeatures from each region to determine which region has a highprobability to be a lesion candidate. This determination is based onfactors such as size, circularity, average gray level, location, margingradient, and the like. Based on the empirical rules of lesion size,shape, margin, and texture features typically used by radiologists tocharacterize a breast lesion in ultrasound image, a breast lesion tendsto be round or oval in shape, and have a stronger gradient along theborder or portion of the border. With the knowledge that lesions tend tobe located around the center of an ultrasound image and with alow-average gray value than its surrounding. The ultrasound image isreviewed for the potential region in each image based on size,circularity, margin gradient, average gray value of each region and itsrelative position in the image. Shown in FIG. 2E is a binary image withthe identified region after morphological smoothing operation to smooththe border.

Selected textures calculated in Equation 3 can be also used to moreparticularly identify a candidate lesion region. A rule-based classifiercan be employed to analyze these features to determine the lesioncandidate having a likelihood of malignancy.

The method of the present invention is directed to providing a method toautomate segmentation of candidate lesions. FIGS. 5A and 5B areultrasound images wherein FIG. 5A shows a manually delineated lesionborder and a computer delineated lesion border. FIG. 5B shows amismatched area between the two borders.

FIGS. 6A-6B, 7A-7B, and 8A-8B show examples of original ultrasoundimages (FIGS. 6A, 7A, and 8A) and the ultrasound image withcomputer-determined contours (FIGS. 6B, 7B, and 8B).

The present invention has been described with the understanding that theultrasound image includes an lesion region. However, it is noted thatthe ultrasound image under review may not comprise a lesion region ormay comprise a lesion region which does not satisfy the thresholddefined by the user. As such, it is possible that no lesion region wouldbe detected, and accordingly, an appropriate message or notation can beprovided to the medical professional.

A computer program product may include one or more storage medium, forexample; magnetic storage media such as magnetic disk (such as a floppydisk) or magnetic tape; optical storage media such as optical disk,optical tape, or machine readable bar code; solid-state electronicstorage devices such as random access memory (RAM), or read-only memory(ROM); or any other physical device or media employed to store acomputer program having instructions for controlling one or morecomputers to practice the method according to the present invention.

All documents, patents, journal articles and other materials cited inthe present application are hereby incorporated by reference.

The invention has been described in detail with particular reference toa presently preferred embodiment, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the invention. The presently disclosed embodiments are thereforeconsidered in all respects to be illustrative and not restrictive. Thescope of the invention is indicated by the appended claims, and allchanges that come within the meaning and range of equivalents thereofare intended to be embraced therein.

1. A method for determining a candidate lesion region in a digitalultrasound medical image of anatomical tissue, the method comprising thesteps of: accessing the digital ultrasound medical image of anatomicaltissue; applying an anisotropic diffusion filter to the ultrasound imageto generate a filtered ultrasound image; performing a normalized cutoperation on the filtered ultrasound image to partition the filteredultrasound image into a plurality of regions; and selecting, from theplurality of regions, at least one region as a candidate lesion region.2. The method of claim 1, wherein the selected candidate lesion regionhas: (1) an intensity value lower than a pre-determined intensity valueand (2) morphological or texture features in accordance withpre-determined lesion criteria.
 3. The method of claim 1, furthercomprising the step of, prior to the selecting step, merging theplurality of regions based on pre-determined threshold values.
 4. Themethod of claim 3, wherein a first threshold value is applied to theplurality of regions having an average gray value lower than apre-determined value G₀, and a second threshold value is applied to theplurality of regions having an average gray value higher thanpre-determined value G₀.
 5. The method of claim 1, further comprisingthe step of segmenting the selected at least one candidate lesion regionin the ultrasound image.
 6. The method of claim 1, wherein the step ofperforming a normalized cut is accomplished by segmenting spatiallycontiguous pixels in the filtered ultrasound image into a plurality ofregions in accordance with substantially similar features.
 7. The methodof claim 6, wherein the normalized cut is accomplished to minimize asimilarity between each of the plurality of regions and maximize asimilarity within each of the plurality of regions.
 8. The method ofclaim 7, wherein the features include intensity-based features and/ortexture-based features.
 9. A method for determining a candidate lesionregion in a digital ultrasound medical image of anatomical tissue, themethod comprising the steps of: accessing the digital ultrasound medicalimage of anatomical tissue; applying an anisotropic diffusion filter tothe ultrasound image to generate a filtered ultrasound image; performinga normalized cut operation on the filtered ultrasound image to partitionthe filtered ultrasound image into a plurality of regions, wherein thenormalized cut is performed by segmenting spatially contiguous pixels inthe filtered ultrasound image into a plurality of regions in accordancewith substantially similar features; merging the plurality of regionsbased on pre-determined threshold values; and selecting, from theplurality of regions, at least one region as a candidate lesion region,wherein the selected candidate lesion region has: (1) an intensity valuelower than a pre-determined intensity value and (2) morphological ortexture features in accordance with pre-determined lesion criteria. 10.The method of claim 9, wherein a first threshold value is applied to theplurality of regions having an average gray value lower than apre-determined value G₀, and a second threshold value is applied to theplurality of regions having an average gray value higher thanpre-determined value G₀.
 11. The method of claim 9, further comprisingthe step of segmenting the selected at least one candidate lesion regionin the ultrasound image.
 12. The method of claim 9, wherein thenormalized cut is accomplished to minimize a similarity between each ofthe plurality of regions and maximize a similarity within each of theplurality of regions.
 13. The method of claim 12, wherein the featuresinclude intensity-based features and/or texture-based features.